Method for setting dominant color using spatial coherency

ABSTRACT

A method is provided for setting a dominant color using a spatial coherency in order to improve performance of an image search function. The method includes describing an image region based on several dominant colors, a frequency of each dominant color, and a spatial coherency (SC) of the dominant color. In the method, the spatial coherency is non-uniformly quantized based on a predetermined threshold. For example, the spatial coherency (SC) may be ranged from 0 to 1 and the threshold may be set to 0.70.

This application is a continuation of U.S. patent application Ser. No.10/995,285, filed Nov. 24, 2004, now U.S. Pat. No. 7,079,683, which is acontinuation of U.S. patent application Ser. No. 10/050,148, filed Jan.18, 2002, now U.S. Pat. 7,006,687, and claims priority to Korean PatentApplication No. 2985/2001 filed Jan. 18, 2001.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to multimedia search. Particularly, thepresent invention relates to a method for setting a dominant color,where the dominant color is included as a feature for searching themultimedia data.

2. Description of the Related Art

In a traditional image search system, it suggests a method for setting adominant color of images that the images are divided into n×m number ofgrids, color histograms are obtained for each divided cell, and arepresentative color having maximum bin of color histogram is designatedas the dominant color of a particular region. In other cases, an averagevalue of color histogram or a major hue vector is selected as thedominant color of the particular region.

To be short, the dominant color can be determined by a method of usingan average color value of a given region; choosing one most frequentlyappeared color; choosing the n most frequently appeared colors; using acolor that prevails in a designated region by more than p%, or using ahistogram, and so forth.

Each method described above has relative advantages and disadvantages inits own way.

For instance, the method using histogram makes it possible to finelyrepresent the color. However, it requires a lot of data and sometimesset the dominant color for colors occupying extremely small part of theregion.

Another method taking a single average value is advantageous that it hasabridged data information and is capable of performing a pre-filteringfunction in a content-based search. Unfortunately however, if the regionis composed of a variety of colors, it is pretty hard to accuratelyrepresent the dominant color.

However, considering that the characteristics of images are usuallydiverse and that in most cases it is difficult to express various colorsby a single color value, the method of expressing an image region usingonly one dominant color is rather inappropriate or insufficient torepresent the entire characteristics of the images and to construct aprecise image characteristic database, so that it causes a deteriorationof a performance of an image search using such a characteristicdatabase.

Moreover, a method of expressing a dominant color of a region based on avariety of colors not only occupies a lot of storage space, but alsomakes it difficult to precisely represent mutual relations among thecolors. In many cases, it is also hard to determine the dominant colorbased on a color distribution of a given image region, and the dominantcolor value obtained by this method lacks precision.

Taking all the aforementioned problems into consideration, inventors ofthe present inventions introduced a new method for describing colorinformation of a given image region, in which the image region isrepresented by several dominant colors and their frequencies (ratio ofthe dominant colors in a given region, percentage %), a spatialcoherency (it is called as a ‘reliability’) for the dominant colors isdefined, and such a reliability is used for the image search togetherwith the dominant colors to improve performance of the image search.This method is described in Korean Patent Application No. 1999-3181(corresponding to U.S. patent application Ser. No. 09/479,888 filed onJan. 10, 2000 and PCT Application No. PCT/KR99/00849 filed on Dec. 31,1999) and Korean Patent Application No. 1999-26784 (corresponding toU.S. patent application Ser. No. 09/609,392 and PCT Application No.PCT/KR00/00710, both filed on Jul. 3, 2000.

Here, the spatial coherency can be a clue or information how much thedominant colors of the given image region are reliable, and it alsoindicates how densely the dominant colors are mixed together in theimage region.

Therefore, when the spatial coherency is defined together with thedominant color, one can find out how reliable the dominant color thereofis at the time of the image search, and based on this, more improvedsearch can be realized.

As a result, it is concluded that performance of the image search isgreatly influenced by how precisely the spatial coherency represents thereliability of the dominant color.

Therefore, it is evident that a simple uniform quantization of thespatial coherency is not sufficient to reflect the diversecharacteristics of the images. Thus, a more improved quantization schemeof the spatial coherency is required.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a method for setting adominant color that substantially obviates one or more problems due tolimitations and disadvantages of the related art.

An object of the present invention to provide a method for setting adominant color using a spatial coherency of the dominant color toimprove performance of a multimedia data search especially when thedominant color is used as a feature of the multimedia data.

To achieve the above object and other advantages and in accordance withthe purpose of the invention, as embodied and broadly described herein,there is provided a method for quantizing spatial coherency of adominant color of an image region that includes non-uniformly quantizinga normalized value for the spatial coherency responsive to arelationship between the normalized spatial coherency value and aprescribed threshold.

To further achieve the above object and other advantages and inaccordance with the purpose of the invention, as embodied and broadlydescribed herein, there is provided a method for quantizing spatialcoherency of at least one dominant color of an image region thatincludes normalizing a weighted sum of per-dominant color-spatialcoherencies from 0 to 1 and non-uniformly quantizing the normalizedspatial coherency to a range from 1 to 31, wherein the normalized valuesof spatial coherency less than 0.7 are set to a quantized value of 1,and wherein the normalized values of spatial coherency from 0.7 to 1 areuniformly quantized to the quantized range from 2 to 31.

To further achieve the above object and other advantages and inaccordance with the purpose of the invention, as embodied and broadlydescribed herein, there is provided a method for describing an imageregion by using at least one dominant color and a spatial coherency onthe at least one dominant color that includes describing a normalizedspatial coherency of at least one dominant color using 31 bins byquantizing normalized values of spatial coherency less than 0.7 to aquantized value of 1, and uniformly quantizing the normalized values ofspatial coherency from 0.7 to 1 to the quantized range from 2 to 31.

It is to be understood that both the foregoing general description andthe following detailed description of the present invention areexemplary and explanatory and are intended to provide furtherexplanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects and features of the present inventionwill become more fully apparent from the following description andappended claims, taken in conjunction with the accompanying drawings.Understanding that these drawings depict only typical embodiments of theinvention and are, therefore not to be considered limiting of its scope,the invention will be described with additional specificity and detailthrough use of the accompanying drawings in which:

FIG. 1 is a flow chart showing a quantization scheme of a spatialcoherency in accordance with a first embodiment of the present inventionfor setting a dominant color;

FIG. 2 is a flow chart showing a quantization scheme of a spatialcoherency in accordance with a second embodiment of the presentinvention for setting a dominant color; and

FIG. 3 is a flow chart showing a quantization scheme of a spatialcoherency in accordance with a third embodiment of the present inventionfor setting a dominant color.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

The following detailed description of the embodiments of the presentinvention, as represented in FIGS. 1-3, is not intended to limit thescope of the invention, as claimed, but is merely representative of thepresently preferred embodiments of the invention. In the description,same drawing reference numerals are used for the same elements even indifferent drawings. The matters defined in the description are nothingbut the ones provided to assist in a comprehensive understanding of theinvention. Thus, it is apparent that the present invention can becarried out without those defined matters. Also, well-known functions orconstructions are not described in detail since they would obscure theinvention in unnecessary detail.

The present invention suggests a method for setting a dominant colorespecially when a dominant color of a given image region and a spatialcoherency (SC) of the dominant color are expressed together, in whichthe method includes steps of comparing the spatial coherency with apredetermined threshold, and non-uniformly quantizing the spatialcoherency by mapping a quantized spatial coherency (QSC) correspondingto the spatial coherency based on the comparison result of the spatialcoherency with the predetermined threshold.

Preferably, the spatial coherency is normalized from real number 0 to 1and the threshold used in the method of the present invention is set to0.70.

If the quantized spatial coherency (QSC) is ‘0’, it means that thespatial coherency is not valid.

On the other hand, if the spatial coherency is smaller than thethreshold, 0.70, the quantization value for the corresponding spatialcoherency is mapped into ‘1’ (QSC =1), while a uniform quantization isapplied to the region having the spatial coherency from 0.70 to 1 asmany as the number of the remaining quantization values. Here, theuniform quantization is performed based on the following formula:Quantized Spatial Coherency (QSC)=(int) [(SC−0.7)/(1.0−0.7)×(2.0^(SC)^(—) ^(BIT)−2.0)+0.5]+2,

wherein the SC_BIT denotes a number of bit assigned to the quantization.

In one embodiment of the present invention, the spatial coherency isquantized into 1 bit, and if the spatial coherency is smaller than thethreshold, 0.70, the QSC becomes zero while, if the spatial coherency isgreater than the threshold, 0.70, the QSC becomes 1.

Preferably, the threshold employed in the present invention may be setby two threshold values, i.e. a first threshold, 0.62, and a secondthreshold, 0.70.

In another embodiment of the present invention, the spatial coherency isquantized into 2 bits or more, and if the spatial coherency at this timeis less than 0.62 (SC<0.62), the QSC becomes zero. On the other hand, ifthe spatial coherency is between 0.62 and 0.70 (0.62≦SC≦0.70), the QSCis 1, and if the spatial coherency is 0.70 and above (SC≧0.70), theuniform quantization is applied to the region having the threshold from0.70 to 1 as many as the number of the remaining quantization values.Similar to the above, the uniform quantization is performed based on thefollowing formula:Quantized Spatial Coherency (QSC)=(int) [(SC−0.7)/(1.0−0.7)×(2.0^(SC)^(—) ^(BIT)−3.0)+0.5]+2,

wherein the SC_BIT denotes a bit number assigned to the quantization andit is two bits or more than two bits (SC_BIT≧2).

The method for setting a dominant color using a spatial coherency is nowexplained with reference to the relevant figures.

As described above, the primary goal of the method is to enhanceperformance of the image search by representing an image region based onseveral dominant colors, their frequencies (percentage of the dominantcolors in a given region) and spatial coherencies corresponding to thedominant colors, and by exploiting the spatial coherencies with thedominant colors during a multimedia data search.

Typically used methods for setting the dominant color includes a methodfor using an average color value of a region, a method for representingthe dominant color by the most frequently appearing color, a method forrepresenting the dominant color by the n most frequently appeared colors(n is a predetermined number), a method for using a color occupying morethan p% in a designated region, or a method using a histogram.

Preferably, the spatial coherency is expressed by a single value forwhole dominant colors of a region or an image (SC: Spatial Coherency forwhole dominant colors).

As mentioned before, the spatial coherency value of the presentinvention is non-uniformly quantized to improve performance of the imagesearch.

Here, the spatial coherency, SC, is obtained based on the followingformula:SC=Σ _(for all i) (COH _(—) Ci×COUNT_(—) PELS _(—) Ci/TOTAL_(—)PELS_OF_(—) R )

wherein, COH_Ci is the per-dominant color coherency of Ci; COUNT_PELS_Ciis the number of pixels of Ci in the region R; and TOTAL_PELS_OF_R isthe size of the region R calculated by counting pixels in the region R.

In order to obtain the spatial coherency, the spatial coherency value isfirst designated to an initial value ‘0’ (Set SC=0), and a variable forcounting the pixel value (SUM_COUNG_PELS) is designated to ‘0’.

After that, VISITED PELp=FALSE is designated for all pixels in the givenregion R, and then COH_Ci and COUNT_PELS_Ci are obtained inconsideration of the whole dominant colors (Ci).

To get final spatial coherency, SC is added by COH_Ci×COUNT_PELS_Ci(i.e., SC=SC+COH_Ci×and is divided by TOTAL_PELS_OF_R (i.e.,SC=SC/TOTAL_PELS_OF_R).

Specifically, the COH_Ci (per-dominant color coherency of Ci) and theCOUNT_PELS_Ci are obtained as follows.

To begin with, a size of a coherency checking mask (CCM) is determined.For example, the CCM is obtained by multiplying CCM_WIDTH (=3) andCCM_HEIGHT (=3). The initial COUNT_PELS_Ci is set zero (0), andTOTAL_NUM_COHERENT is also set zero (0). For the whole pixel PELj in theregion R, if the relation, Ci==COLOR_OF_PELj & VISITED_PELj==FALSE issatisfied, the VISITED_PELj is TRUE (i.e., VISITED_PELj=TRUE), and theCOUNT_PELS_Ci is increased by 1 (i.e., COUNT_PELS_Ci=COUNT_PELS_Ci+1).And, the PELj is adjusted to the center of the CCM for the followingoperations.

Except for the central pixel PELj, if Ci==COLOR₁₃OF₁₃MASKED₁₃PIXELk forall the other masked pixels (MASKED₁₃PIXELk), the value of TOTAL NUMCOHERENT is increased by 1.

Next, the COH_Ci is calculated using the formula,COH_Ci=TOTAL_NUM_COHERENT/COUNT_PELS_Ci/(CCM_WIDTH*CCM_HEIGHT−1).Then,using the obtained COH_Ci and the COUNT_PELS_Ci, the spatial coherencyis obtained based on the formula, Σ_(for all i)(COH_Ci×COUNT_PELS_Ci/TOTAL_PELS_OF _R).

The spatial coherency determined by the above algorithm is normalized bya real number between 0 and 1. The final spatial coherency is thenquantized into 1 bit, 2 bits, 3 bits, 4 bits or 5 bits.

For example, if the spatial coherency is quantized into 1 bit, the QSC(quantized spatial coherency) takes a value of 0 or 1; if the spatialcoherency is quantized into 2 bits, the QSC (quantized spatialcoherency) has a value out of 0, 1, 2 or 3; if the spatial coherency isquantized into 3 bits, the QSC (quantized spatial coherency) has a valueout of 0, 1, 2, 3, 4, 5, 6, or 7; if the spatial coherency is quantizedinto 4 bits, the QSC (quantized spatial coherency) has a value out of 0,1, 2, . . . , 14 or 15; and if the spatial coherency is quantized into 5bits, the QSC (quantized spatial coherency) has a value out of 0, 1, 2,. . . , 30 or 31.

Therefore, the quantization scheme of the spatial coherency is largelycategorized into a uniform quantization scheme in which the spatialcoherency is uniformly divided from 0 to 1 for mapping each quantizedvalue, and a non-uniform quantization scheme in which the sectionbetween 0 and 1 is non-uniformly divided for mapping each quantizedvalue. Preferably, the non-uniform quantization scheme is applied to thepresent invention for improving the search function even when relativelysmall bit numbers are used.

More specifically, the non-uniform quantization scheme of the spatialcoherency according to the present invention is divided into a case ofusing ‘0’ for the quantization value, and a case of excluding (notusing) ‘0’ for the quantization value.

In some cases, ‘0’ is not used for the quantization value because ‘0’ isalready used for implying another, namely, ‘SC is not valid’. Forinstance, when the spatial coherency is quantized into 2 bits, the QSC(quantized spatial coherency) is mappinged over 1, 2 or 3.

FIG. 1 illustrates a first embodiment of the present invention where ‘0’is not used for the quantization value, and the threshold, 0.70, is usedfor the non-uniform quantization.

First of all, the spatial coherency is calculated (S101). Then, thespatial coherency is compared with the threshold, 0.70 (S102). If thespatial coherency is smaller than 0.70, the QSC is mappinged into 1(S103), while if the spatial coherency is greater than the threshold,0.70, a uniform quantization is proceeded to a region having the spatialcoherency from 0.70 to 1 as many as the number of the remainingquantization (S104).

In other words, the QSC is uniformly quantized based on the followingformula (S105):(QSC)=(int) [(SC−0.7)/(1.0−0.7)×(2.0^(SC) ^(—) ^(BIT)−2.0)+0.5]+2

wherein, SC_BIT is the number of bits assigned to the quantization. Forexample, the SC_BIT can be 2 bits, 3 bits, 4 bits or 5 bits.

FIG. 2 and FIG. 3 illustrate methods of quantizing the spatial coherencyusing the quantization value ‘0’ in accordance with the secondembodiment and the third embodiment of the present invention.

If the spatial coherency is quantized using the quantization value 0,the quantized spatial coherency can be expressed in a way that thequantization value ‘0’ is included. For example, if the spatialcoherency is quantized into 2 bits, the quantized spatial coherency(QSC) is mapped into 0, 1, 2 or 3.

Particularly, FIG. 2 illustrates the method of quantizing the spatialcoherency using the quantization value ‘0’ and using 1 bit only.

Referring to FIG. 2, the spatial coherency is first calculated (S201),and is then compared with the threshold, 0.70 (S202).

As a result of the comparison, if the spatial coherency is smaller than0.70, the QSC is mapped into a quantization value of 0. Meanwhile, ifthe spatial coherency is greater than 0.70, the QSC is mapped into aquantization value of 1.

On the other hand, the third embodiment shown in the FIG. 3 shows themethod of obtaining quantized spatial coherency (QSC), especially whenthe spatial coherency is quantized using greater than 1 bit, such as, 2bits, 3 bits, 4 bits or 5 bits.

Again, the spatial coherency is first calculated (S301) like before, butis compared with the thresholds, 0.62 and 0.70 at this time (S302).

If the spatial coherency is smaller than 0.62, the QSC is mapped intothe quantization value 0 (S303), and if the spatial coherency is greateror equal to 0.62 and is smaller than 0.70 (0.62≦SC<0.70), the QSC ismapped into the quantization value 1 (S304).

In the meantime, if the spatial coherency is greater or equal to 0.70,the non-uniform quantization is applied to the region having the spatialcoherency from 0.70 to 1 as many as the number of the remainingquantization (S305).

In other words, the QSC is uniformly quantized based on the followingformula (S305):(QSC)=(int) [(SC−0.7)/(1.0−0.7)×(2.0^(SC) ^(—) ^(BIT)−3.0)+0.5]+2

wherein, SC_BIT is the number of bits assigned to the quantization. Forexample, the SC_BIT can be 2 bits, 3 bits, 4 bits or 5 bits.

As explained before, when a certain image or an image region isrepresented by a dominant color, if the frequency of the dominant colorand the spatial coherency of the dominant color are expressed togetherwith the dominant color, and if the spatial coherency is non-uniformlyquantized, it is possible to improve the performance of image searcheven if the relatively small bit numbers are employed.

For a better image search using information on a dominant color (thedominant color value, the frequency of the color, and the spatialcoherency of the color), a certain image region (or the whole image) canbe compared with another image region. That is, several dominant colorsextracted from each image region, the frequencies of the dominantcolors, and the spatial coherency values on the dominant colors arematched to one another for the comparison.

To do so, the difference, Diff (D1, D2), for reflecting the similaritybetween the two image regions (or the whole images) to be compared iscalculated using the formula:Diff (D1, D2)=W1×SC_Diff×DC_Diff+W2×DC_Diff.In this way, it is searched how similar the two image regions (or thewhole images) are.

In the formula, the W1 denotes a weight on the SC_Diff. For example, theW1 can be designated to 0.3 (W1=0.3). However, if the spatial coherencyis not valid, the W1 is designated to 0 (W1=0). Similarly, the W2denotes a weight on the second item on the right hand side, DC_Diff. Forinstance, W2 can be designated to 0.7 (W2=0.7).

Here, the SC_Diff is an absolute value of the difference of each spatialcoherency of two image regions (or the whole images) to be compared, andthe DC_Diff is the difference between the dominant color values of twoimage regions (or the whole images) to be compared.

In conclusion, the present invention is advantageous for improvingperformance of the search, particularly during the image search, byexpressing the dominant color for an image search together with thefrequency and the spatial coherency thereof, in which the spatialcoherency is non-uniformly quantized based on the predeterminedthreshold. Thus, even if the relatively small bit numbers are employed,the spatial coherency can represent how reliable the dominant color isand it provides more improved performance of image search.

While the invention has been described in conjunction with variousembodiments, they are illustrative only. Accordingly, many alternative,modifications and variations will be apparent to persons skilled in theart in light of the foregoing detailed description. The foregoingdescription is intended to embrace all such alternatives and variationsfalling with the spirit and broad scope of the appended claims.

1. A method for quantizing spatial coherency of at least one dominantcolor of an image region, comprising: normalizing a weighted sum ofper-dominant color-spatial coherencies; and non-uniformly quantizing thenormalized spatial coherency to a predetermined range, wherein thenormalized values of spatial coherency less than a first predeterminedvalue are set to a quantized value of a first predetermined range, andwherein the normalized values of spatial coherency in connection with asecond predetermined value are uniformly quantized to a quantized secondpredetermined range.
 2. The method of claim 1,wherein the weight isproportional to a number of pixels corresponding to each dominant colorof the region.
 3. The method of claim 1, wherein the first predeterminedvalue is 0.62 and the second predetermined value is 0.70.
 4. The methodof claim 3, wherein the normalized values of spatial coherency greaterthan or equal to the first predetermined value and less than or equal tothe second predetermined value are set to a quantized value of a thirdpredetermined range.
 5. The method of claim 1, wherein the normalizedvalues of spatial coherency greater than or equal to the firstpredetermined value and less than or equal to the second predeterminedvalue are set to a quantized value of a third predetermined range. 6.The method of claim 1, wherein the first predetermined value and thesecond predetermined value are the same value.
 7. The method of claim 1,wherein the first predetermined value and the second predetermined valueare 0.70.
 8. A method for quantizing spatial coherency of at least onedominant color of an image region, comprising: normalizing a weightedsum of per-dominant color-spatial coherencies from 0 to 1; andnon-uniformly quantizing the normalized spatial coherency to apredetermined range, wherein the normalized values of spatial coherencyless than a first predetermined value are set to a quantized value of afirst predetermined range, and wherein the normalized values of spatialcoherency in connection with a second predetermined value are uniformlyquantized to a quantized second predetermined range.
 9. The method ofclaim 8, wherein the weight is proportional to a number of pixelscorresponding to each dominant color of the region.
 10. The method ofclaim 8, wherein the first predetermined value is to 0.62 and the secondpredetermined value is 0.70.
 11. The method of claim 10, wherein thenormalized values of spatial coherency greater than or equal to thefirst predetermined value and less than or equal to the secondpredetermined value are set to a quantized value of a thirdpredetermined range.
 12. The method of claim 8, wherein the normalizedvalues of spatial coherency greater than or equal to the firstpredetermined value and less than or equal to the second predeterminedvalue are set to a quantized value of a third predetermined range. 13.The method of claim 8, wherein the first predetermined value and thesecond predetermined value are the same value.
 14. The method of claim8, wherein the first predetermined value and the second predeterminedvalue are 0.70.